The Nuts and Bolts of DBMS Construction: Building your Own Prototype

DBMS construction: DBMS construction: DBMS construction: DBMS construction: DBMS construction: DBMS construction: building your own prototype building your own prototype building your own prototype building your own prototype building your own prototype building your own prototype building your own prototype building your own prototype 2 3 Motivation • www.amazon.com: – 48 books on compiler construction, several on optimizing compilers; – 1 book on DBMS system implementation!! • Researchers wanting to test new concepts are forced to build entire systems from scratch • Students learning how to prototype a DBMS have a hard time finding resources • Haven't we made construction of DBMS any easier in the last 20 years? 4 Goal of the tutorial • To outline techniques for building – advanced research experimental systems – simple but useful query engines • To map resources providing – Building blocks, or – Knowledge on how to build a building block if there is not a satisfactory one at hand 3 5 Method followed in the sequel • Outline a realistic architecture – Identify components and interfaces • Study the interfaces and define ways to combine the components • Use the above components as building-blocks for simple but realistic case studies 6 DBMS architecture Data storage • Data, indices • Materialized views Data and execution statistics Execution engine Optimizer Analyzer Query TEXT Internal query form Query execution plan User Result Catalog schema information 4 7 Recommended prototype design methodology 1. What is the component of interest (the one you want to study / implement / test) ? 2. Where would it fit into a functional system ? (What modules would it interact with ?) 3. Which of these interactions are interesting for you ? 4. How to get (satisfactory approximations of) the modules interacting with yours ? 8 Identifying your target • Already a good step in clarifying ideas • A new component: – A new XML indexing technique – A new (physical) join operator – A new query plan enumeration algorithm • Sometimes a whole system is sought (framework rather than component) – A generic framework for query optimization – A framework for distributed XML management – An XML benchmark 5 9 Identifying your target • Component vs. framework – software engineering issue, with a DB twist 10 Identifying module interactions • Start by considering an "ideal,complete" system architecture • Your component interacts with others – XML index: storage manager, execution …